Image processing apparatus and image pickup device

ABSTRACT

Information of light from a subject is converted into information of the electric charges to generate image data for one screen in an image pickup device, the image data is outputted to an image processing LSI, only the image data belonging to a plurality of areas is selected by a processing amount reduction unit, the selected image data is transferred to a RAM in a single chop by a data transfer unit, and the processing of recognizing a white line is executed by a CPU on the basis of the image data stored in the RAM.

BACKGROUND OF THE INVENTION

The present invention relates to an image processing apparatus and moreparticularly to an image processing apparatus which is suitable forexecuting the arithmetic operation processing for image data stored in awork memory.

The image processing technology for processing image data obtained by animage pickup device has begun to be adapted to the monitoring system fordetecting the invasion of a doubtful person or the abnormality and tothe onboard system for supporting the safety travelling of a motorcar.In order to adapt the image processing technology to these systems, thesmall space and the low cost of an image processing apparatus arerequired. In order to realize the small space and the low cost, there isrequired the device of executing the image processing at high speed withthe limited arithmetic operation ability of a CPU (Central ProcessingUnit) and the limited hardware physical superiority. As for the methodof miniaturizing the image processing apparatus, for example, there isthe method of adopting the single chip. The single chip is such that aCPU for executing the arithmetically operation processing, a ROM (ReadOnly Memory) for storing therein a program, a RAM (Random Access Memory)used as a work memory, and the functions required to construct thesystems such as a network and a serial port are collectively integratedwith each other into a package of one chip. By using this single chip,it becomes possible to realize the system, which is heretoforeconstructed by employing a plurality of devices, in one chip, which cancontribute to the small space and the low cost of the image processingapparatus.

On the other hand, in order to execute the high speed image processingusing a single chip, it is necessary to execute the image processingusing a suitable memory within the single chip. This reason is that anaccess from a CPU to the external memory provided outside the singlechip is generally slower than that to a local memory (work memory)provided in the inside of the single chip. For this reason, whenexecuting the image processing using the external memory, it becomesdifficult to provide the practical throughput. For example, with respectto the white line (traffic line or lane) recognizing function adopted inthe on-board system, the recognition at the video rate is required andhence the high speed processing is required. But, in the case where theprocessing is executed using the external memory, it is difficult tofulfill the required performance. In addition, in the case of anenvironment resistance type single chip for use in the control or thelike of motorcars, since the capacity of a memory in the single chip issmall, an amount of data which can be processed at high speed islimited.

Then, as described in JP-A-2001-101396, there is proposed a methodwherein image data stored in the external memory is sent to a localmemory, which is integrated with a processor into one chip, through theDMA (Direct Memory Access) transfer to process the image data at highspeed. In addition, as described in JP-A-6-4651, there is also proposeda method wherein only the data in an area which is used in theprocessing is transferred in order to execute the image processing athigh speed. Each of those prior arts adopts the construction in whichthe image data is stored in the external memory and the necessary imagedata, when necessary, is transferred to the local memory in order to beprocessed.

SUMMARY OF THE INVENTION

In the former of the above-mentioned prior arts, only the image data,which is required for the image processing, of the image data stored inthe external memory is transferred to the local memory to realize thehigh speed processing.

More specifically, the image data which has been generated by an imagepickup device is inputted to an LSI for image processing through awiring, the image data processed in the image processing LSI istransferred to the external memory through a bus by a data transferunit, and the image data stored in the external memory is transferred tothe local memory in the single chip. In this case, the image data forone screen is transferred to the external memory, and only the necessaryimage data of the image data stored in the external memory istransferred to the local memory in the single chip to execute the imageprocessing at high speed. Thus, there is adopted the method wherein whentransferring the image data to the external memory, the image data forone screen is transferred in its entirety. However, if the recognitionof a white line (traffic line or lane) on a road is taken intoconsideration in the image processing apparatus for on-board, since thewhite lines do not appear in the full screen, it is also conceivable totransfer the image data in the area having the white lines appearingtherein to the external memory. But, in the prior art, the constructionin which the image data for the whole one screen is stored in theexternal memory is adopted by taking into consideration the possibilitythat the area of the white line to be searched may be changed due to theimage processing algorithm or the change in the vehicle position.

However, while in the case of such construction, no problem occurs whenthe external memory has a sufficient capacity, in the apparatus forwhich the low cost is regarded as important, the case where the capacityof the external memory is either insufficient or not present must alsobe considered. At this time, since when the local memory in the singlechip has the insufficient capacity, there is no storage unit for storingtherein image data, it is impossible to construct the image processingsystem. In actual, since the capacity of the local memory is small inthe case of the currently usable single chip, the application is limitedwhich can be made fit for practical use in the image processingapparatus which is loaded with only a local memory as storage means.

On the other hand, while in the latter of the above-mentioned priorarts, only the data used in the image processing is transferred toexecute the processing at high speed, when there is no memory capacityenough to store the data of the original image, it becomes impossible tostore the image data in a memory, and hence it is impossible toconstruct the image processing system.

In the light of the foregoing, the present invention has been made inorder to solve the above-mentioned problems associated with the priorart, and it is therefore an object of the present invention to providean image processing apparatus which is capable of miniaturizing an imagedata storage unit as a local memory without employing any of theexternal memories, and an image pickup device for use in the same.

In order to attain the above-mentioned object, according to one aspectof the present invention, there is provided an image processingapparatus including: an image data storage unit for storing thereinimage data transferred thereto; an arithmetic operation processing unitfor operating arithmetically and processing the image data stored in theimage data storage unit; and a data transfer unit for receiving, as itsinput, the image data and for transferring a plurality of image data, anamount of which is less than that of inputted image data, of the inputimage data to the image data storage unit.

When constructing the above-mentioned image processing apparatus, theremay also adopted the construction in which an image pickup device forconverting information of light from a subject into information of theelectric charges to generate successively image data for one screen isprovided, and also the image data generated by the image pickup deviceis inputted to the data transfer unit. In this case, it is preferablethat the physical storage capacity of the above-mentioned image datastorage unit is set less than an amount of image data outputted from theabove-mentioned image pickup device.

In addition, according to another aspect of the present invention, thereis provided an image processing apparatus including: an image datastorage unit for storing therein image data; an arithmetic operationprocessing unit for operating arithmetically and processing the imagedata stored in the image data storage unit; an image data generationunit for converting information of light from a subject into informationof the electric charges to generate successively image data for onescreen; and a data transfer unit for transferring a plurality of imagedata, an amount of which is less than that of generated image data, ofthe image data generated by the image data generation unit to the imagedata storage unit.

When constructing each of the above-mentioned image processingapparatuses, it is possible to add the following elements.

(1) The data transfer unit transfers, as the plurality of image data,the image data in a plurality of areas on a screen as the object of thearithmetic operation processing in the arithmetic operation processingunit.

(2) The data transfer unit transfers the image data in a plurality ofareas on a screen as the object of the arithmetic operation processingin the arithmetic operation processing unit separately in plural timesevery image data in a single area.

(3) The data transfer unit transfers, as the plurality of image data,the different kinds of image data separately for kinds.

(4) The data transfer unit transfers the image data of a plurality ofedge parts in which the change in concentration within the image isequal to or higher than a fixed level and also transfers coordinateinformation of the plurality of edge parts.

(5) The data transfer unit transfers the image data, of a plurality ofedge parts in which the change in concentration within the image isequal to or higher than a fixed level, of the image data within aplurality of areas on a screen as the object of the arithmetic operationprocessing in the arithmetic operation processing unit, and alsotransfers coordinate information of the plurality of edge parts.

In addition, according to still another aspect of the present invention,there is provided an image pickup device including: an image datageneration unit for converting information of light from a subject intoinformation of the electric charges to generate successively image datafor one screen; and an area selection unit for selecting the image data,belonging to a plurality of areas, of the image data for one screengenerated by the image data generation unit.

When constructing the above-mentioned image pickup device, it ispossible to add the following elements.

(1) The image pickup device further includes a data output unit foroutputting the image data belonging to the plurality of areas selectedby the area selection unit together with area information used tospecify the plurality of areas.

(2) The image pickup device further includes a data transfer unit fortransferring the image data outputted by the data output unit to animage data storage unit for storing therein the image data by either aDMA transfer method or a transfer method based on an SRAM access.

(3) The image pickup device further includes: an exposure time detectionunit for detecting that the exposure time required to generate the imagedata fir one screen has been completed; and a data transfer unit fortransferring the image data outputted by the data output unit to theimage data storage unit for storing therein image data on condition thatthe completion of the exposure time has been detected by the exposuretime detection unit.

(4) The image pickup device further includes a transfer request outputunit for when the completion of the exposure time has been detected bythe exposure time detection unit, outputting a transfer request fortransferring the image data to the destination of transfer.

(5) The image pickup device further includes an image data storage unitfor storing therein the image data generated by the image datageneration unit in correspondence to pixels for one screen, whereinaddresses of the image data stored in the image data storage unit aremapped in such a way as to be associated with a memory space part in thearithmetic operation processing unit for operating arithmetically andprocessing the image data.

(6) An image pickup device includes: an image data generation unit forconverting information of light from a subject into information of theelectric charges to generate successively image data for one screen; andan area selection unit for selecting successively the image data,belonging to a plurality of areas, of the image data for one screengenerated by the image data generation unit every single area.

(7) An image pickup device includes: an image data generation unit forconverting information of light from a subject into information of theelectric charges to generate successively image data for one screen; anarea selection unit for selecting successively the image data, belongingto a plurality of areas, of the image data for one screen generated bythe image data generation unit every single area; and a data transferunit for holding the image data selected by the area selection unit andfor transferring successively the image data thus held every singlearea.

According to the above-mentioned means, a plurality of image data, anamount of which is less than that of input image data, of the image dataused in the arithmetic operation processing is selected before the imagedata is stored in the image data storage unit, and only the image datathus selected is stored in the image data storage unit. Therefore, it ispossible to reduce the storage capacity of the image data storage unitas a local memory (work memory) without employing any of the externalmemories, which can contribute to the small space and the low cost. Inaddition, since an amount of image data stored in the image data storageunit is reduced, the load applied to a bus through which the image datais transferred is reduced, and the high speed processing by thearithmetic operation unit (CPU) becomes possible all the more.Furthermore, the processing of writing the image data to the image datastorage unit can be suppressed to a minimum.

In addition, when transferring the image data, the image data in aplurality of areas on the screen or the image data of a plurality ofedge parts in which the concentration change in the image is equal to orhigher than a predetermined level is transferred, whereby it is possibleto reduce an amount of data. Moreover, the image data of a plurality ofedge parts, in which the concentration change in the image is equal toor higher than a fixed level, of the image data in a plurality of areason the screen is transferred together with the coordinate information,whereby it is possible to reduce an amount of data.

In addition, according to the image pickup device of the presentinvention, since the image data, belonging to a plurality of areas, ofthe image data for one screen is selected to be outputted, it ispossible to increase the transfer speed when transferring the image datagenerated by the image pickup device. Further, the function ofselecting/transferring the image data is added to the image pickupdevice to provide one chip module, which can contribute to the smallspace and the low cost.

Other objects, features and advantages of the invention will becomeapparent from the following description of the embodiments of theinvention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an image processingapparatus according to a first embodiment of the present invention;

FIG. 2 is a conceptual diagram useful in explaining image data used inwhite line recognition;

FIG. 3 is a flow chart useful in explaining the operation of the imageprocessing apparatus shown in FIG. 1;

FIG. 4 is a block diagram showing a configuration of a data transferunit;

FIG. 5 is a flow chart useful in explaining the processing of DMAtransfer;

FIG. 6 is a diagram showing a structure of a memory space part becominga destination of transfer in the DMA transfer;

FIG. 7A is a diagram showing an example of displaying an image of a chipaccording to another embodiment of the present invention;

FIG. 7B is a diagram showing a structure of a memory space partaccording to another embodiment of the present invention;

FIGS. 8A and 8B are respectively diagrams each useful in explaining amethod of reducing a storage capacity utilizing edge parts;

FIG. 9 is a block diagram showing a configuration of another embodimentof a data transfer unit;

FIG. 10 is a block diagram showing a configuration of an imageprocessing apparatus according to a second embodiment of the presentinvention;

FIG. 11A is a diagram useful in explaining the area selection;

FIG. 11B is a block diagram showing a configuration of an area selectionunit;

FIG. 12 is a time chart useful in explaining a structure of output dataoutputted from an image pickup device;

FIG. 13 is a block diagram showing a configuration of an imageprocessing apparatus according to a third embodiment of the presentinvention; and

FIG. 14 is a time chart useful in explaining the operation of the imageprocessing apparatus shown in FIG. 13

DETAILED DESCRIPTION OF THE EMBODIMENTS

An embodiment of the present invention will hereinafter be described indetail with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a configuration of an image processingapparatus for recognizing white lines (traffic lines or lanes) on a roadto identify the position of a driver's vehicle according to oneembodiment of the present invention. For the image processing apparatusas the on-board system, the small space and the low cost are themandatory requirement for the practical use, and also by taking intoconsideration the adoption of the present invention to motorcars forwhich the high safety is required, an environment resistance type singlechip is adopted in the present embodiment. In addition, the imageprocessing apparatus is provided in which the single chip is adopted andthe limited local memory in the single chip is used, whereby the highspeed image processing is realized with the less hardware physicalsuperiority. Hereinbelow, the concrete contents will be described withreference to FIG. 1.

In FIG. 1, the image processing apparatus includes a single chip 1, animage processing LSI 2, and an image pickup device 3. Then, the singlechip 1 is connected to the image processing LSI 2 through a bus 4, andthe image processing LSI 2 is connected to the image pickup device 3.

The image pickup device 3 has the function as an image data generationunit for converting information of light from this world, i.e., lightfrom a subject into information of the electric changes to generatesuccessively digital image data at intervals of 1/60 seconds forexample, and is generally constituted by a sensor including a CCD(Charge Coupled Device) and a CMOS (Complementary Metal-OxideSemiconductor). The image data which has been generated by the imagepickup device 3 is inputted as an image pickup device data output(indicated by an arrow 5) to the image processing LSI 2. The imageprocessing LSI 2 subjects the image data inputted thereto to the imageprocessing, and then a specified plurality of image data of the imagedata which has been obtained after the image processing is selected by aunit 30 for reducing an amount of processing data. The image data to betransferred is selected by the unit 30 for reducing an amount ofprocessing data, whereby the processing of reducing an amount oftransferring data is executed.

The image data which has been selected by the unit 30 for reducing anamount of processing data is transferred to the single chip 1 through abus 4 by a data transfer unit 20 (indicated by an arrow 6). In thiscase, as shown in FIG. 2, the image pickup device 3 generatessuccessively the image data for one screen with respect to a screen 100,and at the time when the image data has been inputted to the imageprocessing LSI 2, the image data belonging to an area A 120 and an areaB 130 each of which is smaller than the screen area 100 istransferred-to the single chip 1 through the bus 4. The setting of thearea A 120 and the area B 130 (indicated by an arrow 7) is determined bya program stored in a ROM 13 in the single chip 1. That is, the controlsignal obedient to this determination is inputted from the single chip 1to the image processing LSI 2 through the bus 4 to set the area A 120and the area B 130.

The single chip 1 includes a CPU 11 acting as an arithmetic operationprocessing unit for operating arithmetically and processing the imagedata, a RAM 12 which is a work memory or a local memory and which actsas an image data storage unit for storing therein the image datatransferred thereto, and a ROM 13 for storing therein a program and thelike. Then, the CPU 11 is adapted to process the image data stored inthe RAM 11 to recognize white lines on a road. The program forrecognizing white lines is stored in the ROM 13. Any of the programsstored in the ROM 13, similarly to the image data stored in the RAM 12,can be accessed at high speed by the CPU 11.

In the image processing apparatus configured as described above, asshown in FIG. 3, first of all, the image pickup device 3 acquires animage on the basis of light from a subject in order to generate theimage data (Step S10), and transfers successively the generated imagedata for one screen to the image processing LSI 2. Then, the imageprocessing LSI 2, prior to the storage of the image data, executes theprocessing of reducing an amount of processing data (Step S11). By theprocessing of reducing an amount of processing data is meant theprocessing of limiting the image data intended to be recorded in the RAM12 within the single chip 1 to the data which is required to beprocessed, and in the present embodiment, is meant the processing ofselecting only the image data belonging to the area A 120 and the area B130 shown in FIG. 2 to transfer the image data thus selected. The imagedata of white lines (that is, traffic lines or lanes) 101 is containedin the area A 120 and the area B 130, respectively, and after completionof the transfer thereof to the single chip 1, is stored in the RAM 12 inthe single chip 1 (Step S12). Thereafter, the CPU 11 starts toarithmetically operate and process the image data stored in the RAM 12to execute the processing of recognizing the white lines 101 (Step S13).

As described above, in the present embodiment, before storage of theimage data in the RAM 12, the image data an amount of which is less thanthat of image data inputted from the image pickup device 3 is selectedto reduce an amount of image data. As a result, it becomes unnecessaryto provide the external memory for storing therein all of the image dataas in the prior art, and hence the system can be constructed by the lesshardware physical superiority, and also it is possible to reduce thehardware physical superiority and the load applied to the bus 4.

By the way, while in the present embodiment, the processing of reducingan amount of processing data and the processing of storing the data areseparetely executed, in actual, the processing is executed in which itis selected whether the input data is the data required to be processedone image data by one image data or is the data not required to beprocessed in such a manner, and after completion of the selection, onlythe selected data is written to the RAM 12. Therefore, the processing ofinterest is the pipeline processing.

Next, the description will hereinbelow be given with respect to aconcrete configuration of the data transfer unit 20 with reference toFIG. 4. In FIG. 4, the data transfer unit 20 has the function oftransferring the image data belonging to the area A 120 and the area B130 shown in FIG. 2 to the RAM 12 in the single chip 1, and alsoincludes an area selection unit A 21 for selecting the image databelonging to the area A 120, and an area selection unit B 22 forselecting the image data belonging to the area B 130. The image dataoutputted from the image pickup device 3 are inputted to each of thearea selection unit A 21 and the area selection unit B 22 through awiring 51. Then, since the area selection unit A 21 and the areaselection unit B 22 have the same function, in this case, thedescription will now be given with respect to only the area selectionunit A 21. The image data which has been inputted through the wiring 51is written to a data buffer 25 in accordance with a write signalcontrolled by a write control unit 24. Now, the write control unit 24holds a circuit for identifying the coordinate position of the imagedata inputted through the wiring 51 and also has the function ofdiscriminating whether or not the inputted image data is the image datawithin the area A 120. Then, if it is discriminated that the inputtedimage data is the image data within the area A 120, then the writecontrol unit 24 permits the write of the image data of interest to thedata buffer 25 so that the write processing is executed. As a result,only the image data within the area A 120 is recorded in the data buffer25. At the time when an equal to or larger than a certain constantamount of image data is stored in the data buffer 25, a request oftransferring the image data to the RAM 12 within the single chip 1 ismade by the data buffer 25 to start the transfer of the image data tothe RAM 12. At this time, a read control unit 23 carries out the readcontrol required for the DMA transfer from the single chip 1 to controlthe processing of reading out the image data stored in the data buffer25. Then, the image data thus read out is transferred to the RAM 12through the DMA transfer.

After the transfer request has been outputted from the data buffer 25 tothe single chip 1, that transfer request is processed as the interruptfor the CPU 11 in the single chip. The processing flow at this time willnow be described with reference to FIG. 5.

First of all, after the transfer request has been outputted from thedata buffer 25 to the CPU 11 in the single chip 1, in response to thattransfer request, the CPU 11 generates an interrupt (Step S20). Then,the CPU 11 in the single chip 1 analyzes the interrupt factor orcondition (Step S21). That is, the CPU 11 analyzes whether or not thetransfer request is inputted from the data buffer 25 due to the factthat the number of image data stored in the data buffer 25 has becomeequal to or larger than the constant amount. When it is analyzed thatthe transfer request is inputted from the data buffer 25 due to the factthat the number of image data stored in the data buffer 25 has becomeequal to or larger than the constant amount, an output address of thedata buffer 25 is set as an address of transfer source (Step S22). Next,an address of transfer destination of the data is set (Step S23). Inthis case, as shown in FIG. 6, a memory space A 121 in a memory spacepart 140 of the RAM 12 is assumed as an area in which the data in thearea A 120 is stored, and the head address of the memory space A 121 isset as the address of transfer destination. After completion of thesetting of the address of transfer source and the setting of the addressof transfer destination, the transfer number of times of DMA transfer isdetermined from the total number of the data stored in the data buffer25 to set the number of times of transfer (Step S24). Finally, the DMAtransfer is activated (Step S25) to DMA-transfer the image data in thedata buffer 25 to the memory space A 121 in the RAM 12 in the singlechip 1. Those processings are repeatedly executed until the imageprocessing of the image data for one screen has been completed.

By the way, in the case as well where the data buffer of the areaselection unit B 22 makes the transfer request, the DMA transfer iscarried out on the basis of the same processing flow. This processing isrepeatedly executed until the completion of the processing of the imagedata for one screen.

According to the present embodiment, since only the image data requiredfor the processing is transferred to the RAM (local memory) 12 withinthe single chip 1, it is possible to reduce the storage capacity for thedata. In addition, since an amount of transferring data is also small,it is possible to reduce the load applied to the bus, and also it ispossible to prevent the reduction of the throughput of the CPU 11. Inaddition, since the image processing apparatus includes a plurality ofdata transfer units, the data in a plurality of areas can be transferredto the memory space part for the purpose of being readily arithmeticallyoperated. This result in that it is unnecessary to execute theprocessing of rearranging the data, and the like in the processing ofrecognizing while lines after completion of the data transfer, and canalso contribute to the reduction of an amount of arithmetic operations.

While in the present embodiment, the description has been given withrespect to the image data, belonging to the two areas, as the image datato be transferred to the RAM 12 within the single chip 1, it is alsopossible to adopt the configuration of transferring the image databelonging to equal to or larger than three areas. Now, the descriptionwill hereinbelow given with respect to an embodiment when transferringthe image data belonging equal to or larger than three areas withreference to FIGS. 7A and 7B. In the embodiment shown in FIGS. 7A and7B, the inspection of leads 310 of a semiconductor part is supposed, andthe inspection of the leads 310 is the application for which though awide field of view is employed, the high measurement accuracy isrequired. An image processing apparatus for inspecting the leads 310, inorder to enhance the measurement accuracy, is adapted to acquire animage 300 in the form of a high definition image. This image 300contains a marker 311 which is formed on a chip 301 in order to alignthe chip 301, and the leads 310, as the objects of the inspection, whichare extracted from an end part of the chip 301. In the case of thisinspection, the image data of the whole screen is not used, but only theimage data belonging to areas 320, 321, 322 and 323 having thepossibility of the presence of the marker 311 and the leads 310 is used.Those areas are not fixed because they are dispersed depending on time,lots and parts, and hence need to be set by a program.

In the present embodiment, memory spaces 341 342, 343 and 344corresponding to the areas 320, 321, 322 and 323 used in the imageprocessing are ensured on a memory space part 340 in the RAM 12. Then,the image data belonging to the respective areas is transferred as theimage data required for the image processing to the RAM 12 on the singlechip 1 by data transfer units corresponding to the areas. In this case,the CPU 11 recognizes the length of each of the leads 310 to check thequality of the leads 310.

According to the present embodiment, since though the high definitionimage data is used, only the image data required for the processing istransferred, it is possible to record the image data required for theprocessing in the RAM with the small storage capacity. In addition, theimage data belonging to an arbitrary area can be transferred bycontrolling the range of the area used for the transfer by a program.Furthermore, since a plurality of data transfer units are provided inthe areas, the transfer units can be used in an overlap manner, andhence the present embodiment can be applied to the flexible application.

As described above, according to the present embodiment, the imageprocessing apparatus having only the necessary minimum storage capacitycan be applied to the application as well using the high definitionimage for which the large storage capacity is required if normal.

While in each of the above-mentioned embodiments, the description hasbeen given with respect to the method wherein the reduction of an amountof data stored in the RAM 12 is realized by limiting the area to beprocessed, in the following embodiment, the description will hereinbelowbe given with respect to a method of reducing an amount of datautilizing information of edge parts of a white line as shown in FIGS. 8Aand 8B.

As shown in FIG. 8A, when recognizing white lines on the basis of imagedata, belonging to an area 110, of image data for one screen outputtedby the image pickup device 3, if the image data used to recognize whitelines 102 is subjected to the filtering processing for contourenhancement, then it becomes possible to extract the both sides of thewhite line 102 in the form of edge parts. That is, it becomes possibleto extract the image data of the edge parts in which the concentrationchange (luminance change) in the image is equal to or larger than afixed level. Since the processing of recognizing the white line 102 isexecuted by utilizing the edge information which the white line 102 has,only the information (image data) of the edge parts is used in theprocessing in many cases. If only this information of the edge parts isused in the recognition processing, then an amount of data to be storedmay be an amount of data which is less than amount of all of the imagedata belonging to the area 110 is stored. Further, as for theinformation to be recorded, the coordinate information of the positionswhich are determined to be the edge parts may also be used. In FIG. 8B,the data transfer of the coordinate information used to specify the areaA 120 and the area B 130 is carried out in combination with the datatransfer of the image data of the edge parts, i.e., the different kindsof information is transferred separately for kinds, whereby it ispossible to reduce further the storage capacity for the data requiredfor the processing. A concrete configuration of the data transfer unit20 at this time is shown in FIG. 9.

The data transfer unit 20 includes a preprocessing unit 400 in additionto an area selection unit A 21 and an area selection unit B 22. The datatransfer unit 20 is adapted to subject the image data which has beeninputted through a wiring 53 in order to recognize the white lines tothe filtering processing for contour enhancement in the preprocessingunit 400 to extract the edge parts to transfer only the information(image data) of the edge parts to the RAM (local memory) 12 within thesingle chip 1 using the area selection unit A 21 and the area selectionunit B 22. At this time, for the area selection unit A 21, the same oneas the area selection unit A 21 shown in FIG. 4 may be employed. But,the function of determining whether or not not only the image data ofinterest is the image data of the coordinates of the input image data,but also the image data of interest is the image data of the edge partsthereof is added to the write control unit 24.

According to the present embodiment, the image data is transferred onthe assumption that only the image data of the edge parts is used in theprocessing, whereby it is possible to reduce the storage capacityrequired for the image processing apparatus. In addition, the area asthe object of the processing is limited on the screen, whereby it ispossible to obtain the effect for the combination of the reduction of anamount of data due to the limitation of the area and the method oflimiting the information to the information of the edge parts. In otherwords, it is possible to reduce further the storage capacity, for theimage data, required for the processing.

While in each of the above-mentioned embodiments, the image pickupdevice 3 has been handled as the device for outputting simply the imagedata, it is possible to adopt the configuration in which the unit 30 forreducing an amount of processing data shown in FIG. 1 is added to theimage pickup device 3. An embodiment at this time will hereinbelow bedescribed with reference to FIG. 10.

Referring now to FIG. 10, in the image processing apparatus according tothe present embodiment, an area selection unit 510 is provided in theimage pickup device 3, and a data transfer unit 500 is provided in theimage processing LSI 2. That is, the image pickup device 3 has thefunctions as an image data generation unit for converting information oflight from a subject into information of the electric charges togenerate successively the image data for one screen, an area selectionunit for selecting the image data, belonging to a plurality of areas, ofthe image data thus generated, and a data output unit for outputting theimage data thus selected belonging to a plurality of areas together withthe area information used to specify the areas through a bus 520.

The area selection unit 510, as shown in FIGS. 11A and 11B, executes theprocessing of setting an area A 650 and an area B 651, and when settingthe area A 650 belonging to the screen 100, sets the data of a startingpoint coordinate 600, a width 601 and a height 602 of the area A 650,while when setting an area B 651, sets the data of a starting pointcoordinate 610, a width 611 and a height 612 of the area B 651. Afterthe data of the starting point coordinates, the width and the height ofeach of the areas A 650 and the area B 651 has been set, the informationof the area A 650 and the area B 651 is recorded in an area storageunits 620 and 621 shown in FIG. 11B, respectively. The data recorded inthe area storage units 620 and 621 is read out by a sensor data readunit 630 to be outputted together with the area information used tospecify the areas through a wiring 632. In this case, in the imagepickup device 3, the sensor data read unit 630 having the function ofoutputting the image data stored in the area storage unit 620 hascontrol of the output data.

To put it concretely, an array of photoelectric receiving devices in aCMOS sensor is likened to an SRAM, and under this condition, a circuitfor an SRAM access may be added to the sensor data read unit 630. Atthis time, the processing of reading out the data is executed with thecoordinate values as an address. In addition, the control may also becarried out in such a way that the output signals from the CMOS sensorare classified into the signal which is required to be outputted to theoutside of the image pickup device 3 and the signal which is notrequired to be outputted thereto in the sensor data read unit 630 tooutput only the necessary data. Further, both of the valid data and theinvalid data may also be outputted to output the signal used to identifywhether the outputted data is valid or invalid.

For example, in the case where the data of a plurality of areas isoutputted from the image pickup device 3, as shown in a time chart ofFIG. 12, whenever the valid data signal 521 is outputted, output data522 as the image data belonging to an area A is outputted together witharea data 503 used to specify the area A, and also image data (outputdata) 522 belonging to an area B is outputted together with area data523 used to specify the area B. Now, the area data 523 used to specifythe area A and the area B may be the information used to determine towhich DATA a and DATA b each as the output data 522 belong, and hencethe information such as the coordinate information or an ID of an areamay be employed for the area data 523.

As described above, in the present embodiment, when only the image data,belonging to the specific area, of the image data generated by the imagepickup device 3 is outputted to the image processing LSI 2 through thewiring 520 to be transferred to the single chip 1 through the bus 4 bythe data transfer unit 500 in the image processing LSI 2, the data whichis required to be recorded in the RAM 12 in the single chip 1 isselected in the image pickup device 3. As a result, the image processingLSI 2 has only to include the function of transferring the data.

Therefore, according to the present embodiment, the function ofselecting data is additionally provided in the image pickup device 3,whereby it is possible to read out only the data used in processing athigh speed.

Next, the description will hereinbelow be given with respect to anembodiment when the data transfer unit 500 provided in the imageprocessing LSI 2 is additionally provided in the image pickup device 3with reference to FIG. 13.

The image pickup device 3 in the present embodiment has the functions ofa data transfer unit 500 and an area selection unit 510. The imageprocessing apparatus according to the present embodiment is constitutedby the single chip 1 and the image pickup device 3, and hence can beminiaturized as compared with the image processing apparatus accordingto the above-mentioned embodiment. Then, the image pickup device 3 andthe single chip 1 are connected to each other through a bus 4, and theregister and the like of the image pickup device 3 are mapped on thememory space part of the single chip 1. At this time, the register ofthe image pickup device 3 corresponds to the area storage unit 620 shownin FIG. 11B. In addition, each of the photoelectric receiving devicesincluded in the image pickup device 3 is mapped on the memory space partof the single chip 1 so that the image pickup device 3 may also behandled as the external memory. That is, the addresses of the image datastored in the RAM 12 can be mapped in such a way as to be associatedwith the memory space part of the CPU 11. This becomes possible byadding either the circuit for making the DMA transfer possible or thecircuit described with reference to FIG. 11B.

The transfer of the image data from the image pickup device 3 in thepresent embodiment to the RAM 12 of the single chip 1, as shown in FIG.14, is carried out on condition that completion of an exposure period oftime 700 has been detected. That is, when taking in the image atintervals of 1/60 to 1.0 seconds to generate successively the image datafor one screen, the image pickup device 3 generates the image datawhenever the exposure period of time 700 has elapsed. Then, an exposuretime detection unit for detecting the exposure period of time 700 isprovided in the image pickup device 3. At the time when the exposureperiod of time detection unit has detected the completion of theexposure time 700, the image pickup device 3 generates an interrupt 701to the single chip 1. In response to the interrupt 701 from the imagepickup device 3, the CPU 11 in the single chip 1 instructs the imagepickup device 3 to start the DMA transfer 702, and then the DMA transferof the image data from the image pickup device 3 to the RAM 12 in thesingle chip 1 is started on the basis of the processing by the datatransfer unit 500. In this case, when an interrupt is generated everyline, the DMA transfer is generated every line. In addition, when aninterrupt is generated every field, the DMA transfer is activated everyfield. By the way, a timing chart shown in FIG. 14 may be defined forevery line of the image, or may be defined for every field. Inparticular, in the case of a CMOS sensor, it is general in the operationof the current CMOS sensor that FIG. 14 is viewed as a timing chart foreach of lines.

According to the image pickup device 3 of the present embodiment, sincethe image pickup device 3 is provided with the function of transferringthe image data of the specified area in the screen to be formed intoone-chip module, this can contribute to the small space and the lowcost.

While in each of the above-mentioned embodiments, the description hasbeen given with respect to the procedure in which in the transfer of theimage data, the image data of a plurality of areas is transferred atonce, there may also be adopted the procedure in which the image data ofa plurality of areas is selected from the generated image data for onescreen, the selected image data is divided into a plurality of singleareas, and the image data in each of the single areas is transferredseparately in plural times.

In addition, there may be adopted the procedure in which a plurality ofimage data of a single area is selected from the generated image datafor one screen, and the selected image data is successively transferredseparately in plural times. In this case, that procedure can be realizedby storing the processing program for transferring the image data of asingle area separetely in plural times. For example, that procedure canbe realized by executing the processing program to set an area havingimage data to be transferred to a memory separately in plural times inthe image data for one screen obtained through one exposure to transfersuccessively the image data from the image pickup device to the memoryin the single chip.

In addition, when the image data is transferred separately in pluraltimes every single area, as for the area selection unit for the datatransfer unit, there may be employed a single area selection unit forselecting a single area plural times.

Furthermore, transferring the image data, in a plurality of areas, ofthe image data for one screen separately in plural times means that theimage pickup device serves as the memory too and hence the image pickupdevice holds the image data until the transfer of all of the image datagenerated by the image pickup device has been completed.

As set forth hereinabove, according to the present invention, beforestorage of image data in an image data storage unit, a plurality ofimage data, an amount of which is less than that of input-image data, ofthe image data used in the arithmetic operation processing is selected,and only the selected image data is stored in an image data storageunit. Therefore, it is possible to reduce the storage capacity of theimage data storage unit as a local memory without employing any of theexternal memories, which can contribute to the small space and the lowcost.

It should be further understood by those skilled in the art that theforegoing description has been made on embodiments of the invention andthat various changes and modifications may be made in the inventionwithout departing from the spirit of the invention and the scope of theappended claims.

1. An image processing apparatus comprising: image data storage meansfor storing therein image data transferred thereto; arithmetic operationprocessing means for arithmetically processing the image data stored insaid image data storage unit; and data transfer means for receiving, asits input, image data and for transferring a plurality of image data, anamount of which is less than that of inputted image data, of the inputimage data to said image data storage unit, wherein said data transfermeans transfers the image data in a plurality of areas on a screen asthe object of the arithmetic operation processing in said arithmeticoperation processing means separately in plural times every image datain a single area.
 2. An image processing apparatus comprising: imagedata storage means for storing therein image data transferred thereto;arithmetic operation processing means for arithmetically processing theimage data stored in said image data storage unit; and data transfermeans for receiving, as its input, image data and for transferring aplurality of image data, an amount of which is less than that ofinputted image data, of the input image data to said image data storageunit, wherein said data transfer means transfers, as the plurality ofimage data, the different kinds of image data separately for kinds. 3.An image processing apparatus comprising: image data storage means forstoring therein image data transferred thereto; arithmetic operationprocessing means for arithmetically processing the image data stored insaid image data storage unit; and data transfer means for receiving, asits input, image data and for transferring a plurality of image data, anamount of which is less than that of inputted image data, of the inputimage data to said image data storage unit, wherein said data transfermeans transfers the image data of a plurality of edge parts in which thechange in concentration within the image is equal to or higher than afixed level and also transfers coordinate information of the pluralityof edge parts.
 4. An image processing apparatus comprising: image datastorage means for storing therein image data transferred thereto;arithmetic operation processing means for arithmetically processing theimage data stored in said image data storage unit; and data transfermeans for receiving, as its input, image data and for transferring aplurality of image data, an amount of which is less than that ofinputted image data, of the input image data to said image data storageunit, wherein said data transfer means transfers the image data, of aplurality of edge parts in which the change in concentration within theimage is equal to or higher than a fixed level, of the image data withina plurality of areas on a screen as the object of the arithmeticoperation processing in said arithmetic operation processing means, andalso transfers coordinate information of the plurality of edge parts. 5.An image pickup device comprising: image data generation means forconverting information of light from a subject into information of theelectric charges to successively generate image data for one screen;area selection means for selecting the image data, belonging to aplurality of areas, of the image data for one screen generated by saidimage data generation means; and data output means for outputting theimage data belonging to the plurality of areas selected by said areaselection means and area information used to specify the plurality ofareas, wherein the area information and the image data are output viadifferent signal lines.
 6. An image pickup device comprising: image datageneration means for converting information of light from a subject intoinformation of the electric charges to generate successively image datafor one screen; area selection means for selecting the image data,belonging to a plurality of areas, of the image data for one screengenerated by said image data generation means; data output means foroutputting the image data belonging to the plurality of areas selectedby said area selection means together with area information used tospecify the plurality of areas; and exposure time detection means fordetecting that the exposure time required to generate the image data forone screen has been completed; and data transfer means for transferringthe image data outputted by said data output means to said image datastorage means for storing therein image data on condition that thecompletion of the exposure time has been detected by said exposure timedetection means.
 7. An image pickup device according to claim 6, furthercomprising transfer request output means for when the completion of theexposure time has been detected by said exposure time detection means,outputting a transfer request for transferring the image data to thedestination of transfer.
 8. An image pickup device comprising: imagedata generation means for converting information of light from a subjectinto information of the electric charges to generate successively imagedata for one screen; and area selection means for selecting the imagedata, belonging to a plurality of areas, of the image data for onescreen generated by said image data generation means; and image datastorage means for storing therein the image data generated by said imagedata generation means in correspondence to pixels for one screen,wherein addresses of the image data stored in said image data storagemeans are mapped in such a way as to be associated with a memory spacepart in said arithmetic operation processing means for operatingarithmetically and processing the image data.